The effect of prior experience on diagnostic reasoning: exploration of availability bias.

Objectives Diagnostic reasoning has been shown to be influenced by a prior similar patient case. However, it is unclear whether this process influences diagnostic error rates or whether clinicians at all experience levels are equally susceptible. The present study measured the influence of specific prior exposure and experience level on diagnostic accuracy. Methods To create the experience of prior exposure, participants (pre-clerkship medical students, emergency medicine residents, and faculty) first verified diagnoses of clinical vignettes. The influence of prior exposures was measured using equiprobable clinical vignettes; indicating two diagnoses. Participants diagnosed equiprobable cases that were: 1) matched to exposure cases (in one of three conditions: a) similar patient features, similar clinical features; b) dissimilar patient features, similar clinical features; c) similar patient features, dissimilar clinical features), or 2) not matched to any prior case (d) no exposure). A diagnosis consistent with a matched exposure case was scored correct. Cases with no prior exposure had no matched cases, hence validated the equiprobable design. Results Diagnosis A represented 47% of responses in condition d, but there was no influence of specific similarity of patient characteristics for Diagnosis A, F(3,712)=7.28, p=0.28 or Diagnosis B, F(3,712)=4.87, p=0.19. When re-scored based on matching both equiprobable diagnoses, accuracy was high, but favored faculty (n=40) 98%, and residents (n=39) 98% over medical students (n=32) 85%, F(2,712)=35.6, p<0.0001. Accuracy for medical students was 84, 87, 94, and 73% for conditions a-d, respectively, interaction F(2,712)=3.55, p<0.002. Conclusions The differential diagnosis of pre-clerkship medical students improved with prior exposure, but this was unrelated to specific case or patient features. The accuracy of medical residents and staff was not influenced by prior exposure.

Reflecting on Diagnostic Errors: Taking a Second Look is Not Enough.

BACKGROUND: An experimenter controlled form of reflection has been shown to improve the detection and correction of diagnostic errors in some situations; however, the benefits of participant-controlled reflection have not been assessed. OBJECTIVE: The goal of the current study is to examine how experience and a self-directed decision to reflect affect the accuracy of revised diagnoses. DESIGN: Medical residents diagnosed 16 medical cases (pass 1). Participants were then given the opportunity to reflect on each case and revise their diagnoses (pass 2). PARTICIPANTS: Forty-seven medical Residents in post-graduate year (PGY) 1, 2 and 3 were recruited from Hamilton Health Care Centres. MAIN MEASURES: Diagnoses were scored as 0 (incorrect), 1 (partially correct) and 2 (correct). Accuracies and response times in pass 1 were analyzed using an ANOVA with three factors-PGY, Decision to revise yes/no, and Case 1-16, averaged across residents. The extent to which additional reflection affected accuracy was examined by analyzing only those cases that were revised, using a repeated measures ANOVA, with pass 1 or 2 as a within subject factor, and PGY and Case or Resident as a between-subject factor. KEY RESULTS: The mean score at pass 1 for each level was PGY1, 1.17 (SE 0.50); PGY2, 1.35 (SE 0.67) and PGY3, 1.27 (SE 0.94). While there was a trend for increased accuracy with level, this did not achieve significance. The number of residents at each level who revised at least one diagnosis was 12/19 PGY1 (63 %), 9/11 PGY2 (82 %) and 8/17 PGY3 (47 %). Only 8 % of diagnoses were revised resulting in a small but significant increase in scores from Pass 1 to 2, from 1.20/2 to 1.22 /2 (t = 2.15, p = 0.03). CONCLUSIONS: Participants did engage in self-directed reflection for incorrect diagnoses; however, this strategy provided minimal benefits compared to knowing the correct answer. Education strategies should be directed at improving formal and experiential knowledge.

Disrupting diagnostic reasoning: do interruptions, instructions, and experience affect the diagnostic accuracy and response time of residents and emergency physicians?

PURPOSE: Others have suggested that increased time pressure, sometimes caused by interruptions, may result in increased diagnostic errors. The authors previously found, however, that increased time pressure alone does not result in increased errors, but they did not test the effect of interruptions. It is unclear whether experience modulates the combined effects of time pressure and interruptions. This study investigated whether increased time pressure, interruptions, and experience level affect diagnostic accuracy and response time. METHOD: In October 2012, 152 residents were recruited at five Medical Council of Canada Qualifying Examination Part II test sites. Forty-six emergency physicians were recruited from one Canadian and one U.S. academic health center. Participants diagnosed 20 written general medicine cases. They were randomly assigned to receive fast (time pressure) or slow condition instructions. Visual and auditory case interruptions were manipulated as a within-subject factor. RESULTS: Diagnostic accuracy was not affected by interruptions or time pressure but was related to experience level: Emergency physicians were more accurate (71%) than residents (43%) (F = 234.0, P < .0001) and responded more quickly (54 seconds) than residents (65 seconds) (F = 9.0, P < .005). Response time was shorter for participants in the fast condition (55 seconds) than in the slow condition (73 seconds) (F = 22.2, P < .0001). Interruptions added about 8 seconds to response time. CONCLUSIONS: Experienced emergency physicians were both faster and more accurate than residents. Instructions to proceed quickly and interruptions had a small effect on response time but no effect on accuracy.

The mediating effect of context variation in mixed practice for transfer of basic science.

Applying a previously learned concept to a novel problem is an important but difficult process called transfer. Practicing multiple concepts together (mixed practice mode) has been shown superior to practicing concepts separately (blocked practice mode) for transfer. This study examined the effect of single and multiple practice contexts for both mixed and blocked practice modalities on transfer performance. We looked at performance on near transfer (familiar contexts) cases and far transfer (unfamiliar contexts) cases. First year psychology students (n = 42) learned three physiological concepts in a 2 × 2 factorial study (one or two practice contexts and blocked or mixed practice). Each concept was practiced with two clinical cases; practice context was defined as the number of organ systems used (one system per concept vs. two systems). In blocked practice, two practice cases followed each concept; in mixed practice, students learned all concepts before seeing six practice cases. Transfer testing consisted of correctly classifying and explaining 15 clinical cases involving near and far transfer. The outcome was ratings of quality of explanations on a 0-3 scale. The repeated measures analysis showed a significant near versus far by organ system interaction [F(1,38) = 3.4, p < 0.002] with practice with a single context showing lower far transfer scores than near transfer [0.58 (0.37)-0.83 (0.37)] compared to the two contexts which had similar far and near transfer scores [1.19 (0.50)-1.01 (0.38)]. Practicing with two organ contexts had a significant benefit for far transfer regardless of mixed or blocked practice; the single context mixed practice group had the lowest far transfer performance; this was a large effect size (Cohen's d = 0.81). Using only one practice context during practice significantly lowers performance even with the usually superior mixed practice mode. Novices should be exposed to multiple contexts and mixed practice to facilitate transfer.

Ineffectiveness of cognitive forcing strategies to reduce biases in diagnostic reasoning: a controlled trial.

OBJECTIVES: Cognitive forcing strategies (CFS) may reduce error arising from cognitive biases. This is the first experimental test to determine the effect of CFS training in medical students. METHODS: Students were allocated to CFS training or control during a 4-week emergency medicine rotation (n  =  191). At the end of the rotation examination, students were tested using computer-based cases. Application of CFS could enable reduction of diagnostic error, as evidenced by identifying multiple correct diagnoses for the two cases prone to search satisficing bias (SSB) and uncommon diagnoses for the two cases prone to availability bias (AB). Two "false positive" cases were included to test for possible "oversearching." RESULTS: There were 145 students in the intervention and 46 in the control group. For the SSB cases, 52% of students with CFS training and 48% in the control group initiated a search for the second diagnosis (χ2  =  0.13, df  =  1, p  =  0.91). More than half (54%) correctly identified the second diagnosis in the CFS group, and 48% identified it in the control group. The difference was not significant (χ2  =  2.25, df  =  1, p  =  0.13). For the second diagnosis in the false positive cases, 64% of the CFS group and 77% of the control group incorrectly identified it. There were no significant differences between groups (χ2  =  2.38, df  =  1, p  =  0.12). In the AB cases, only 45% in each group identified the uncommon correct diagnosis (χ2  =  0.001, df  =  1, p  =  0.98). CONCLUSIONS: The educational interventions suggested by experts in clinical reasoning and employed in our study to teach CFS failed to show any reduction in diagnostic error by novices.

The effect of conceptual and contextual familiarity on transfer performance.

Applying a previously learned concept to a novel problem is an important but difficult process called transfer. It is suggested that a commonsense analogy aids in transfer by linking novel concepts to familiar ones. How the context of practice affects transfer when learning using analogies is still unclear. This study investigated the effect of a commonsense analogy and context familiarity for transfer of physiological concepts. First year psychology students (n = 24) learned three concepts: Starling's law, Laplace's law, and laminar-turbulent flow. The control group saw standard explanations while the intervention group saw an additional commonsense analogy. The context of learning was the organ system used for two practice clinical cases which differed for all concepts. Testing consisted of 12 new clinical cases. Starling's law cases used the organ system from practice while the other concepts presented in both novel and familiar organ systems. Half of the sample repeated testing after 1 week delay. The outcome was ratings of explanations of cases on a 0-3 scale. The effect of analogy was significant (Mean = 1.24 with, 0.86 without, F(1,22) = 4.26, p < 0.05) but not after delay (means of 1.08 and 0.75 respectively, F = (1,10), p = 0.06) There was significant effect for familiar context (Same = 1.23 (Starling), different = 0.68 (Laplace) and 0.73 (laminar-turbulent flow) (F(2,44) = 5.14, p < 0.01). Laplace's law and laminar turbulent flow cases in the familiar organ system had means of 1.65 and 1.77 respectively compared to novel cases with means of 0.74 and 0.68 (F(1,22) = 35.64, p < 0.0001). Similar effects were observed after delay. There was significant decay in performance after delay for all participants (immediate = 1.17, delayed = 0.91, F = 11.9 (1,10) p < 0.01). Common analogies aid conceptual understanding necessary for transfer. Despite conceptual aids, solving transfer problems is difficult.

Speed kills? Speed, accuracy, encapsulations and causal understanding.

BACKGROUND: The role of basic science, which provides causal explanations for clinical phenomena in medical education, is poorly understood. Schmidt has postulated that expert clinicians maintain this knowledge in 'encapsulated' form, indexed by words or phrases describing the processes. In the present paper we show that students who learn causal explanations have a more coherent understanding of the relation between diseases and clinical features which, in turn, influences recognition of words or phrases describing 'encapsulated knowledge' and the ability to maintain performance under speeded conditions. HYPOTHESES: In comparison to students who simply learn the features of 4 diagnostic categories, students who learn a causal explanation will: (a) recognise words describing encapsulated knowledge more accurately and (b) maintain or improve diagnostic performance under speeded conditions. METHODS: Two studies were conducted involving 4 'pseudo-endocrinology' diseases and undergraduate psychology students. One group learned signs and symptoms alone; the second group also learned a causal explanation. In study 1, they were then given a recognition memory task. In study 2, they were asked to diagnose new cases either (i) as quickly as possible or (ii) taking their time. RESULTS: In study 1, while there was no difference in recognising old words (90% versus 91%), the causal group was better able to recognise encapsulated and novel consistent words (50% versus 41%) (P = 0.02). In study 2 there was an interaction; causal students performed better under speeded conditions (71% versus 66%) but worse under thoroughness conditions (67% versus 73%), as predicted. CONCLUSIONS: Causal understanding leads to more coherent understanding of clinical conditions, which in turn leads to expert-like behaviour.

The value of basic science in clinical diagnosis.

BACKGROUND: The role of basic science knowledge in clinical diagnosis is unclear. There has been no experimental demonstration of its value in helping students recall and organize clinical information. This study examines how causal knowledge may lead to better recall and diagnostic skill over time. METHOD: Undergraduate medical students learned either four neurological or rheumatic disorders. One group learned a basic science explanation for the symptoms. The other learned epidemiological information. Both were then tested with the same set of clinical cases immediately after learning and one week later. RESULTS: On immediate test, there was no difference in accuracy (70% for both groups). However, one week later, performance in the epidemiology group dropped to 51%; the basic science group only dropped to 62%. CONCLUSIONS: Basic science knowledge relating causal knowledge to disease symptoms can improve diagnostic accuracy after a delay.